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Abstract 

Emotion dysregulation is associated with attention deficit/hyperactivity disorder (ADHD) and confers risk for behavior 
problems and functional impairment; however, there is little guidance on best practices for measurement in adolescents. 
We developed a parent-report version of the Difficulties in Emotion Regulation Scale (DERS-P)). Evidence of reliability and 
validity was evaluated in a large community online sample (Study |: n = 978; M,,. = 13.52 years; SD = 1.93) and in two 
samples of adolescents with ADHD (Study 2, Sample |: n = 78; M,,. = 12.12 years, SD = 0.91; Sample 2: n = 206; M,,. 
= 15.35 years; SD = 0.85). A four-factor solution of the DERS-P was obtained in Study | and confirmed in Study 2, with 
factors demonstrating acceptable internal consistency. The community sample was generally rated as less dysregulated 
than the ADHD samples. Support was obtained for convergent, concurrent, and incremental validity evidence. These 
findings provide preliminary evidence for the DERS-P as a psychometrically sound parent-report measure of emotion 


dysregulation in | 1- to 17-year-old adolescents. 
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Despite variability across definitions, emotion regulation can 
be conceptualized as an individual’s ability to modulate (1) 
the speed with which and degree to which the physiological, 
experiential, and behavioral expression of an emotion esca- 
lates; (2) the intensity of the physiological, experiential, and 
behavioral expression of an emotion; and (3) the speed with 
which and degree to which physiological, experiential, and 
behavioral expression of an emotion deescalates in a manner 
congruent with an optimal level of functioning (Bunford, 
Evans, & Wymbs, 2015, p. 188). Emotion dysregulation 
(ED) is an individual’s inability to exercise any or all aspects 
of the emotion regulation modulatory processes, to such a 
degree that the inability results in the individual failing to 
successfully adapt to and meet environmental challenges and 
goals and thus functioning meaningfully below his or her 
baseline (Bunford, Evans, & Wymbs, 2015, p. 188). 

ED is a transdiagnostic characteristic associated with 
externalizing (e.g., oppositional defiant disorder), internal- 
izing (e.g., anxiety, depression), and, as defined here, neu- 
rodevelopmental disorders (Bunford, Evans, & Wymbs, 
2015; Mazefsky et al., 2013) such as attention deficit/ 
hyperactivity disorder (ADHD; Bunford, Evans, Becker, & 


Langberg, 2015; Bunford, Evans, et al., 2017; Bunford, 
Evans, & Langberg, 2014; Graziano & Garcia, 2016; 
Okado & Mueller, 2016; Sjéwall, Backman, & Thorell, 
2015). Although whether ED is a core feature or merely an 
associated feature of ADHD remains debated (Barkley, 
2010), there is a growing body of work indicating that ED 
is associated with negative outcomes such as alcohol prob- 
lems (Bunford, Wymbs, Dawson, & Shorey, 2017) and 
smoking (Mitchell et al., 2012), and with functional impair- 
ment in the academic and social domains (Bunford, Evans, 
Becker, et al., 2015; Maedgen & Carlson, 2000; McQuade, 
Penzel, Silk, & Lee, 2016; Melnick & Hinshaw, 2000) 
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among children, adolescents, and adults with ADHD. 
These findings underscore the importance of ED in asso- 
ciation with ADHD across the life span; however, there is 
little guidance on best practices for measuring ED in this 
population, especially in adolescents (Bunford, Evans, 
et al., 2017; Bunford, Evans, & Wymbs, 2015). 

ED in children with ADHD has been evaluated using mul- 
tiple methodologies, ranging from the most micro to the most 
macro level (Morris & Cuthbert, 2012). Examples include 
genetic (Merwood et al., 2014) physiological (Bunford, 
Evans, et al., 2017; Musser et al., 2011; Musser, Galloway- 
Long, Frick, & Nigg, 2013), observational (C. Lee et al., 
2017; Maedgen & Carlson, 2000; Melnick & Hinshaw, 
2000), parent report and self-report on rating scale (Seymour, 
Chronis-Tuscano, Iwamoto, Kurdziel, & MacPherson, 2014) 
and ecological momentary assessment (Rosen & Epstein, 
2010; Rosen, Epstein, & Van Orden, 2013) methods. 

Conversely, considerably less is known about ED in ado- 
lescents with ADHD and what is known is based exclu- 
sively on studies wherein ED was assessed via self-report 
on rating scales (Bunford et al., 2014; Bunford, Evans, 
Becker, et al., 2015; Seymour et al., 2012). These limita- 
tions are problematic, for the following reasons. First, 
upward extensions of findings obtained with children to 
information about adolescents is unfounded, in part because 
the development of emotion processing and regulation 
capacities during adolescence has unique aspects and chal- 
lenges. Relative to children, adolescents experience more 
intense and labile emotions (Silk, Steinberg, & Morris, 
2003), during a period when asynchronicity between brain 
development, behavioral, cognitive, and emotional self- 
regulatory skill development, and environmental demands 
for self-regulation is at its peak (Steinberg, 2005). Deficits 
or delays in acquisition of emotion processing and regula- 
tion skills can thus be especially debilitating for adoles- 
cents, with this debilitation likely exacerbated in the context 
of psychiatric disorders or symptoms. For example, in the 
context of ADHD, above and beyond symptoms, a number 
of manifestations of ED have been shown to predict comor- 
bidities and functional impairments (Bunford et al., 2014; 
Bunford, Evans, Becker, et al., 2015; Seymour et al., 2014). 
In addition, in youth with ADHD, there are some differ- 
ences and similarities in the aspects of ED that are relevant 
to children relative to adolescents. For example, emotional 
lability, negativity, and reactivity are less prevalent among 
younger than in older youth (Graziano & Garcia, 2016) but 
across children and adolescents, emotional inflexibility and 
slow return to baseline have been shown to predict indices 
of observed and parent- and self-reported social impairment 
(Bunford et al., 2014; Melnick & Hinshaw, 2000). 

Second, evident from their definitions, emotion regula- 
tion and dysregulation are multidimensional phenomena, 
both in terms of the corresponding experiential or phenom- 
enological experience and of the observable or outward 


manifestation. Yet the noted exclusive reliance in adoles- 
cents on self-report precludes comprehensive measurement 
of ED. Despite the importance of using comparable assess- 
ment methods across two or more respondents for gaining 
contextual information (i.e., contexts in which a character- 
istic does or does not occur; De Los Reyes et al., 2015) and 
such methods being considered best-practice in evaluating 
mental health in youth (Youngstrom, Choukas-Bradley, 
Calhoun, & Jensen-Doss, 2015), the absence of valid mea- 
sures of ED across informants precludes parallel assess- 
ment across adolescents and their parents (Mash & Hunsley, 
2005). Furthermore, because of increases in cognitive 
maturity and the internal nature of emotions, self-report 
may be an appropriate method for measuring adolescent ED 
(Rohrbeck, Azar, & Wagner, 1991; Soto, John, Gosling, & 
Potter, 2008; Walden, Harris, & Catron, 2003). However, 
youth with ADHD exhibit a substantial response bias and 
tendency to use a dichotomous and positively skewed 
response style when reporting their emotional experiences 
and regulation (Rosen et al., 2013), challenging the view 
that self-report is sufficient, especially for adolescents with 
the disorder (see, e.g., Maedgen & Carlson, 2000; Melnick 
& Hinshaw, 2000, for evidence of observable behavioral 
correlates of ED in youth with ADHD). Rather, to comple- 
ment data obtained via self-report measures on the experi- 
ential experience of ED, development and evaluation of 
parallel parent-report measures that capture observable 
manifestations of ED is needed. 

One thoroughly evaluated and widely used self-report 
measure of ED is the Difficulties in Emotion Regulation 
Scale (DERS; Gratz & Roemer, 2004). Although the DERS 
was developed and its psychometric properties were initially 
evaluated with adult samples (Gratz & Roemer, 2004), evi- 
dence for acceptable psychometric properties of the measure 
has been established in a variety of adolescent samples, 
including good internal consistency, test-retest reliability, 
adequate construct, and predictive validity (Adrian et al., 
2009; Bunford et al., 2014; Bunford, Evans, Becker, et al., 
2015; Vasilev, Crowell, Beauchaine, Mead, & Gatzke-Kopp, 
2009; Weinberg & Klonsky, 2009), with robust correlations 
with psychological problems reflecting ED (Weinberg & 
Klonsky, 2009) and with physiological measures of ED 
(Vasilev et al., 2009). There are additional advantages of the 
DERS, including that the measure is based on an integrative 
conceptualization of emotion regulation, one that also 
encompasses the definition adopted in the current study. 
Specifically, that conceptualization involves not only the 
modulation of emotional arousal and its various physiologi- 
cal, experiential, and behavioral expressions but also the 
acceptance, awareness, and understanding of emotions, as 
well as ability to act in desired ways even in the face of 
strong emotions (Gratz & Roemer, 2004). Findings also sup- 
port the transdiagnostic utility of the measure, insofar as 
they indicate that the DERS or subscales predict disorder 
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status (e.g., generalized anxiety disorder: Salters-Pedneault, 
Roemer, Tull, Rucker, & Mennin, 2006; posttraumatic stress 
disorder: McDermott, Tull, Gratz, Daughters, & Lejuez, 
2009; Tull, Barrett, McMillan, & Roemer, 2007), past child- 
hood maltreatment (Soenke, Hahn, Tull, & Gratz, 2010), 
nonsuicidal self-injury (Perez, Venta, Garnaat, & Sharp, 
2012), and depression (Ehring, Fischer, Schniille, Bésterling, 
& Tuschen-Caffier, 2008). 

Despite these advantages to the DERS, there is some con- 
cern regarding the stability of its factor structure, potentially 
due to problems with one of its factors that consists of 
reverse-coded items (Bardeen, Fergus, Hannan, & Orcutt, 
2016; Bardeen, Fergus, & Orcutt, 2012; D. J. Lee, Witte, 
Bardeen, Davis, & Weathers, 2016). In the first study on the 
factor structure of the DERS (Gratz & Roemer, 2004), 
although a six- and a seven-factor solution was also identi- 
fied, the authors retained the six-factor solution, as it was 
more interpretable: Awareness (six items; indicative of 
awareness or acceptance of one’s emotions); Clarity (five 
items; indicative of clarity or knowledge about one’s emo- 
tions); Goals (five items; indicative of difficulty with goal- 
directed behavior when upset); Impulse (six items; indicative 
of difficulty with behavioral control when experiencing 
negative emotions); Nonacceptance (six items; indicative of 
negative secondary emotions); and Strategies (eight items; 
indicative of a belief that no strategy can decrease negative 
emotion). Similarly, Weinberg and Klonsky (2009) found 
evidence for both a six- and a seven-factor solution and 
chose to retain the six-factor solution, arguing that the sev- 
enth factor may have represented an artifact of item format 
rather than content. In an attempt to determine a source of 
these problems related to the factor structure of the DERS, 
Bardeen and colleagues made note of and tested problems 
with the Awareness subscale that comprises only reverse- 
coded items (Bardeen et al., 2012; Bardeen et al., 2016; D. J. 
Lee et al., 2016). Bardeen et al. (2016) forward coded all 
reverse-coded items, submitted them to exploratory factor 
analysis (EFA) in one sample, and then to confirmatory fac- 
tor analysis (CFA) in another sample and identified five fac- 
tors, evidence of potential problems with the Awareness 
subscale and the reverse-coded items. 

Despite these inconsistencies across these studies, there 
is a body of research that, although not unequivocally and 
not with regard to all aspects of psychometric properties, 
does support the six-factor structure of the DERS, includ- 
ing use of the Awareness subscale and reverse-coded items 
(Coutinho, Ribeiro, Ferreirinha, & Dias, 2010; Hervas, 
2011). This line of research includes studies conducted 
with large samples across cultures and languages 
(Mitsopoulou, Kafetsios, Karademas, Papastefanakis, & 
Simos, 2013 [nm = 780]; Neumann, van Lier, Gratz, & Koot, 
2010 [n = 870]) as well as across racial groups (n = 
Caucasian 446, African American 424, and Asian American 
180) and sexes (n = male 256 and female 794; Ritschel, 


Tone, Schoemann, & Lim, 2015). Beyond considerations 
regarding factor structure, the Awareness subscale is mean- 
ingfully, albeit sometimes less strongly (e.g., Neumann 
et al., 2010) correlated with the other DERS subscales 
(Fowler et al., 2014; Giromini, Velotti, De Campora, 
Bonalume, & Cesare Zavattini, 2012; Perez et al., 2012; 
Ritschel et al., 2015) and is related to relevant outcomes, 
including delinquency (Neumann et al., 2010), differentiat- 
ing cocaine-dependent patients and community-controlled 
volunteers (Fox, Axelrod, Paliwal, Sleeper, & Sinha, 2007) 
and differentiating individuals with alcohol dependence 
during abstinence from social drinkers (Fox, Hong, & 
Sinha, 2008). 

Taken together, although the results of some studies do 
bring into question the established factor structure of the 
self-report DERS and raise questions about the Awareness 
subscale, the results of other studies provide evidence sup- 
porting the validity of that factor structure. Extended exam- 
ination of the psychometric properties of this self-report 
measure of ED in adolescents, including to cross-informant 
consistency and convergent and incremental validity would 
bolster confidence in these findings. Simultaneously, devel- 
opment and evaluation of parallel measures would fulfill 
the need to establish reliable and valid tools to assess ED 
that do not rely exclusively on self-report and thus increase 
our ability to gather multi-informant data on an important 
transdiagnostic construct. 


Current Study 


To begin addressing the gap in the literature, we developed 
a parent-report form of the DERS (DERS-P; Gratz & 
Roemer, 2004)'. Despite potential problems with the self- 
report version, initiating a line of research examining a 
parent-report form is warranted in light of the considerable 
clinical promise and utility of such parent-rated version. 
Our aim in the current research was to evaluate the psycho- 
metric properties—including evaluating evidence for reli- 
ability and validity—of the DERS-P among adolescents 
with and without ADHD. 

Our first two aims were to examine (1) the factor struc- 
ture and (2) the internal consistency of the DERS-P. These 
aims are addressed in Study | using a large online national 
sample of parents of adolescents. Subsequently, we wanted 
to examine the evidence of (3) the convergent validity of the 
DERS-P as indexed by its association with two self-report 
measures of ED, (4) the concurrent validity of the DERS-P 
as indexed by its association with theoretically relevant 
constructs such as anxiety and depression, and (5) the incre- 
mental validity of the DERS-P as indexed by the degree to 
which it predicts, beyond the self-report DERS, outcomes 
associated with ED. These aims were addressed in Study 2 
using a community sample and a sample of parents of ado- 
lescents with ADHD and their children. 
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We predicted that (1) the self-report and parent-report 
forms will not have the same factor structure given that cer- 
tain aspects of ED are directly observable, some can only be 
inferred, and others are largely internal. This hypothesis is 
consistent with differences in the psychometric properties 
of parent- and child-report versions of rating scales of other 
characteristics. For example, others have reported a differ- 
ent factor structure between parent- and child-report ver- 
sions of measures of callous unemotional traits (Gao & 
Zhang, 2016) and of health quality of life (Steele, 2008). As 
the current research was aimed at evaluating the psycho- 
metric properties of the DERS-P, our design was explor- 
atory and thus we did not formulate hypotheses about the 
other aspects of such properties— internal consistency or 
convergent, concurrent, and incremental validity. 


Study |: Exploration of Factor 
Structure 


Method 


Participants. Participants were 978 Amazon’s Mechanical 
Turk (1.e., MTurk; www.mturk.com) workers who reported 
to be U.S. residents and parents of youth aged 11 to 17 
years. There were participants from each of the 50 states 
and the District of Columbia. Participants were predomi- 
nantly married (80%) and mothers (57.8%). Although most 
participants were White (77.3%), other racial groups were 
represented (8.4% Black, 5.2% Latinx, and 5.0% Asian). 
Annual total family income was assessed in ranges (in 
US$), and were generally well distributed with most parents 
reporting income considered generally within the middle 
class (23.1% made $25,001-45,000, 21.0% made $45,001- 
65,000, 21.6% made $65,001-85,000, 15.3% made $85,001- 
105,000, 8.2% made $105, 001-125,000, 4.9% made 
$125,001-145,000, with the remainder 5.9% making 
>$145,001 annually). Concerning the mother (or female 
guardian) of the child, 1.1% did not complete high school, 
13.3% had high school diploma, 18.8% obtained some col- 
lege but no degree, 17.8% received additional technical 
training of 2-year degree, 33.5% obtained a 4-year degree, 
and 13.7% obtained higher than a 4-year degree. Concern- 
ing the father (or male guardian) of the child, 2.1% did not 
complete high school, 14.6% had high school diploma, 
14.8% obtained some college but no degree, 16.6% received 
additional technical training of 2-year degree, 33.8% 
obtained a 4-year degree, and 12.2% obtained higher than a 
4-year degree. Thus, a greater percentage of parents reported 
having obtained higher educational degree than the general 
population (29.8% of adults >25 years of age report having 
a 4-year degree or higher; U.S. Census Bureau, 2016). 
Youths’ ages ranged from 11 to 17 years (M = 13.52; SD 
= 1.93). There was a slightly higher percentage of boys 
(51.2%), and like parents, most adolescents were White 


(72.4%), with other racial groups represented (8.0% Black, 
5.4% Latinx, 4.3% Asian, and 8.0% multiracial). A small 
percentage (12.1%) of parents indicated that their child has 
special needs eligible for school-based services. Of these 
youth, 55.1% had specials needs due to a neurodevelop- 
mental disorder (e.g., autism spectrum disorder, ADHD), 
10.2% due to academic problems (e.g., learning disorder, 
speech delay), 22.05% due to medical problems (e.g., cystic 
fibrosis, diabetes), and 29.7% due to emotional and behav- 
ioral disorders (e.g., anxiety, depression), not exclusively. 


Procedures. Participants were recruited through MTurk, the 
largest online crowdsourcing platform used frequently and 
effectively for (clinical) research (Chandler & Shapiro, 
2016). On MTurk, individuals (i.e., referred to as workers) 
can complete tasks for compensation (Sheehan & Pittman, 
2016). Although MTurk samples are not as representative 
of the general population as national probability samples, 
they are more representative of the U.S. population than 
typical convenience samples (e.g., college or community 
samples) and workers produce reliable and valid data 
(Chandler & Shapiro, 2016; Sheehan & Pittman, 2016). 

MTurk qualifiers were used to limit eligible respondents 
to individuals who were U.S. residents and had worker 
approval ratings of =85% (an MTurk index of the quality of 
participants’ responding). Study measures were combined 
into a single survey, which was advertised as an academic 
survey about emotion regulation. To qualify for survey 
completion, respondents first had to provide digital 
acknowledgment of informed consent. Then, they under- 
went a brief eligibility screener, wherein they were asked 
whether they had siblings, were married, and were a parent 
(and, if so, the ages of their children). Questions about sib- 
lings and marriage were used as distractor questions so that 
respondents would not be aware of qualifying answers. In 
sum, 3,715 individuals consented to participate; of these, 
90.2% reported that they had siblings, 58.7% reported they 
were married, and 65.3% reported that they were parents. 
Of parents, 43.8% reported having children between the 
ages 11 and 17 years, thus passing initial eligibility. Of 
these, 979 completed the survey. On examination of 
responses, one respondent was deleted for reporting that he 
or she was not a resident of the United States. Data collec- 
tion was completed within approximately 16 hours after 
posting the survey to MTurk. All measures were parent- 
report scales. Participants who completed the survey were 
compensated $2.00 through MTurk payments. This research 
was approved by the Ohio University Social—Behavioral 
Institutional Review Board. 


Measures. The DERS-P was developed by rewording items 
of the original self-report DERS, which, as noted, has dem- 
onstrated evidence of reliability and validity in adult and 
adolescent samples (Gratz & Roemer, 2004; Weinberg & 


Bunford et al. 


Klonsky, 2009). The measure was adapted to assess paren- 
tal perceptions of adolescent ED without a specified time 
frame, similar to the self-report measure. For example, the 
item on the self-report DERS, “I am clear about my feel- 
ings” was changed on the DERS-P to “My child is clear 
about his/her feelings.” Items are rated on a 5-point scale (1 
= almost never, 5 = almost always), with higher scores 
indicating greater ED. The factor structure of the DERS-P 
will be evaluated in this study. In the current sample, the 
internal consistency across all 36 DERS-P items was accept- 
able (a = .84). 


Data Analytic Plan. To address Aim 1, we examined the num- 
ber and content of the latent factors that represent items 
included in the DERS-P. Procedures consistent with evi- 
dence-based guidelines for scale development (DeVellis, 
2016) and identical to those employed with the self-report 
DERS (Gratz & Roemer, 2004) were followed. Accord- 
ingly, we conducted an EFA using all 36 items, using prin- 
cipal axis factoring with promax oblique rotation with 
Kaiser normalization. We excluded items with poor or dual 
loadings (<.40 on any subscale or >.40 on two or more 
subscales) and further relied on criteria indicating that items 
with a loading of less than .32 as well as with cross-loadings 
less than .15 difference from an item’s highest factor load- 
ing should be eliminated (Brady, Evans, Berlin, Bunford, & 
Kern, 2012; Worthington & Whittaker, 2006). 

To determine factor retention, we examined eigenvalues 
above | and the elbow in the Scree Plot (Costello & 
Osborne, 2005). To confirm the EFA results based on the 
Kaiser—Guttman criterion, parallel analysis and the mini- 
mum average partial test were conducted. After deletion of 
an item, the EFA was conducted again until no items were 
indicated for deletion. Additionally, to address Aim 2, each 
subscale was assessed for evidence of internal consistency 
(considered acceptable if a = .70; Nunnally, 1978). 


Results and Discussion 


Results of the first EFA indicated seven items (i.e., 4, 5, 9, 
17, 20, 31, 34; see Table 1 for item descriptions) to be 
deleted given inadequate loading on any extracted factor 
(i.e., <.40) through the EFA iterations. Deleting these items 
also made conceptual sense, as they predominantly pertain 
to adolescents’ beliefs or thoughts (e.g., believes, is con- 
fused, has no idea, has difficulty making sense out of), 
which are difficult for any observer, including a parent, to 
assess, and would thus contribute to greater rater-specific 
bias (i.e., rater-specific error within each respective item 
unrelated to the latent factor). 

In the final EFA, suitability for factor analysis was exam- 
ined (Kaiser-Meyer—Olkin [KMO] = 0.959; Bartlett’s test 
= 17,734.29, p < .001), and indicated the data were appro- 
priate for factoring. Examining the eigenvalues, four factors 


had an eigenvalue >1.0 and accounted for 57.49% of the 
variance. On examination of the elbow, the scree plot also 
appeared to indicate a four-factor solution, and all items 
adequately loaded onto a factor (see Table 1). Results of the 
parallel analysis suggested a three-factor solution and those 
of the minimum average partial test suggested a four-factor 
solution. It should be noted that although the parallel analy- 
sis suggested a three-factor solution, the results fell just 
below the cut-off for suggesting a four-factor solution (i.e., 
for the four-factor solution, the eigenvalue for the sample 
was 1.2347, whereas the eigenvalue for the simulative sam- 
ple was 1.2387). Given that both the Kaiser—Guttman crite- 
rion and the minimum average partial tests suggested a 
four-factor solution and the parallel analysis was just below 
the cut-off for suggesting a four-factor solution, a decision 
was made to select the four-factor solution. 

In comparison with the adolescent self-report 36-item 
scale, two fewer factors emerged in the parent-report ver- 
sion. The original Subscales 3 (Impulse) and 5 (Strategies, 
with exception of Item 30) include items related to the abil- 
ity to maintain control over behavioral expression of emo- 
tions even in face of strong emotions or to implement 
strategies to reduce negative emotions—now loaded onto 
the DERS-P Factor 1. We labeled this factor Catastrophize, 
as many items are related to losing control over and feeling 
overwhelmed by negative emotions. Most of the items on 
the self-report DERS that loaded onto Subscale 1 
(Nonacceptance) loaded onto the DERS-P Factor 2 (with 
the addition of Item 30). The Nonacceptance subscale is 
indicative of a tendency to exhibit negative secondary emo- 
tions (Gratz & Roemer, 2004). Accordingly, we labeled this 
factor Negative Secondary. All items that remained from 
the self-report DERS Subscales 4 (Awareness) and 6 
(Clarity), representative of awareness or clarity of emo- 
tions, loaded onto the DERS-P Factor 3. We labeled this 
factor Attuned (with greater scores reflecting less awareness 
or clarity of emotions). Finally, all remaining items from the 
self-report DERS Subscale 2 (Goals) loaded onto the 
DERS-P Factor 4. These items are related to difficulty with 
engaging in goal-directed behavior (e.g., concentrating, 
focusing, and completing tasks) in the face of strong emo- 
tions. As such, we labeled this factor Distracted. All factors 
demonstrated acceptable internal consistency: Factor 1 
(Catastrophize; 12 items; a = .93), Factor 2 (Negative 
Secondary; 7 items; a = .88), Factor 3 (Attuned; 6-items; a 
= .88), and Factor 4 (Distracted; 4 items; a = .90). 


Study 2: Validation of Factor Structure 
Method 


Participants. In addition to the national online sample of 
parents described above, this study also included two sam- 
ples of adolescent participants with ADHD (n = 284 
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Table |. DERS-P Items and Factor Loadings Following Exploratory Factor Analysis. 


Factor Loadings 


OF Items (DERS-P) | 2 3 4 

6 |. My child is clear about his/her feelings 034 065 696 -.023 
4 2. My child pays attention to how he/she feels -.065 -.02 836 04 
3 3. My child experiences his/her emotions as overwhelming and out of control .764 036 -.087 -.089 
6 4. My child has no idea how he/she is feeling —_ — _— — 
6 5. My child has difficulty making sense out of his/her feelings _— — _ — 

4 6. My child is attentive to his/her feelings -.001 -.019 .792 -.009 
6 7. My child knows exactly how he/she is feeling -.010 .007 759 «116 
4 8. My child cares about what he/she is feeling -.008 020 .7I1 -.074 
6 9. My child is confused about how he/she feels _— —_— —_ —_— 

4 10. When my child is upset, he/she acknowledges his/her emotions -.005 016 .690 -.055 
| 11. When my child is upset, he/she becomes angry with him/herself for feeling that way .289 533° -.040 -.077 
| 12. When my child is upset, he/she becomes embarrassed for feeling that way 062 .732 -.041 -.066 
2 13. When my child is upset, he/she has difficulty getting work done .780 
3 14. When my child is upset, he/she becomes out of control 954 -.076 -.003 -.130 
5 15. When my child is upset, he/she believes that he/she will remain that way for a long time 536 167 -.061 122 
5 16. When my child is upset, he/she believes that he/she will end up feeling very depressed -466 321  -.030 .002 
4 17. When my child is upset, he/she believes that his/her feelings are valid and important —_— — —_— _— 

2 18. When my child is upset, he/she has difficulty focusing on other things -.018 -.073 -.002 910 
3 19. When my child is upset, he/she feels out of control 821 026 -.032 035 
2 20. When my child is upset, he/she can still get things done — —_ — — 

| 21. When my child is upset, he/she feels ashamed with him/herself for feeling that way -.073 831 016 -.037 
5 = 22. When my child is upset, he/she knows that he/she can find a way to eventually feel better 497 -.049 318 -.041 
| 23. When my child is upset, he/she feels like he/she is weak 109 560 .070 039 
3 24. When my child is upset, he/she feels like he/she can remain in control of his/her behaviors .503 -.090  .257 029 
| 25. When my child is upset, he/she feels guilty for feeling that way -.181 OTL = =.031 =-.051 
2 26. When my child is upset, he/she has difficulty concentrating —.112 .028 -.008 -946 
3 27. When my child is upset, he/she has difficulty controlling his/her behaviors 796 -.070 .027 056 
5 28. When my child is upset, he/she believes that there is nothing he/she can do to make him/ 567 188 029 052 


herself feel better 


| 29. When my child is upset, he/she becomes irritated with him/herself for feeling that way 022 -634  .009 114 
5 30. When my child is upset, he/she starts to feel very bad about him/herself 132 617  .026 085 
5 31. When my child is upset, he/she believes that wallowing in it is all he/she can do —_— — —_— _— 

3 32. When my child is upset, he/she loses control over his/her behaviors 857 -.065 -.024 -.0I7 
2 33. When my child is upset, he/she has difficulty thinking about anything else 183 001 -.010 681 
4 34. When my child is upset, he/she takes time to figure out what he/she is really feeling — — — _— 

5 35. When my child is upset, it takes him/her a long time to feel better 507 056 .034 165 
5 36. When my child is upset, his/her emotions feel overwhelming 502 062 -.044 323 


Note. DERS-P = Difficulties in Emotion Regulation Scale—Parent-report form. OF = original factor and indicates what factor each item loaded onto for the 
adolescent self-report version of the DERS (i.e., | = Nonacceptance; 2 = Goals; 3 = Impulse; 4 = Awareness; 5 = Strategies; 6 = Clarity). Boldface font 


indicates final factor solution. 


participants and their primary caretaker). The two samples 
were obtained from two larger, multisite treatment studies, 
including a sample of middle school students with ADHD 
(see Evans et al., 2016; hereafter “middle school ADHD”) 
and a sample of high school students with ADHD (see 
Owens et al., 2018, November; hereafter “high school 
ADHD”).’ In the middle school ADHD sample (n = 78),° 
youth ranged in age from 10 to 14 years (VM = 12.12, SD = 
0.91) and were in Grades 6th through 10th (/ = 6.96; SD 
= 0.78). They were predominantly boys (75.6%), 


non-Hispanic (98.7%), and White (89.7%). In the high 
school ADHD sample (” = 206), youth ranged in age from 
13 to 17 years (M = 15.35; SD = 0.85) and were in Grades 
9th through 11th (MM = 9.75; SD = 0.76) in high school. 
They were predominantly boys (78.2%), non-Hispanic 
(89.4%), and White (72.9%; black: 22.4%). 


Procedures. Data were collected in the context of larger 
research studies, in the middle school ADHD sample, data 
were collected at posttreatment, in the high school ADHD 
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sample, data were collected at eligibility evaluations/pre- 
treatment. Data were combined across the two studies for 
purposes of confirming the factor structure of the DERS-P. 
Otherwise, data from the three available samples were ana- 
lyzed separately to evaluate evidence of convergent, con- 
current, and incremental validity of the DERS-P and its 
indicated factors. 


Measures. In addition to the DERS-P, additional parent- 
and self-report measures were included to assess evidence 
of validity for the DERS-P. In some cases, because different 
measures of the same characteristic were used across stud- 
ies (e.g., for reasons of age-appropriateness or respective 
research design), some measures were administered only to 
the online sample, middle school ADHD sample, or high 
school ADHD sample. These differences are noted below. 

The Beck Youth Inventory [Anxiety (BYI-II-Anxiety; 
Beck, Beck, Jolly, & Steer, 2005) is a 20-item self-report 
scale of anxiety symptoms administered to youth 7 to 18 
years old. Each item is rated on a 4-point scale (0 = never; 
3 = always), with higher scores indicative of greater anxi- 
ety. The BYI-I]-Anxiety has demonstrated evidence of reli- 
ability and validity, including strong association with other 
self-report measures of anxiety (Beck et al., 2005). In the 
current research, the BYI-[I-Anxiety demonstrated excel- 
lent internal consistency in the high school ADHD sample 
(a = .91). 

See above for item content and prior research on the psy- 
chometric properties of the DERS (Gratz & Roemer, 2004). 
The DERS has six subscales (as described above): 
Awareness (six items), Clarity (five items), Goals (five 
items), Impulse (six items), Nonacceptance (six items), and 
Strategies (eight items). Items are rated on a 5-point scale (1 
= almost never; 5 = almost always), with higher scores 
indicative of greater ED. The DERS subscales demon- 
strated adequate internal consistency in the middle school 
ADHD and the high school ADHD samples across domains, 
respectively (i.e., Awareness: a = .61, .74; Clarity: a = .62, 
.74; Goals: a = .79, .82; Impulse: a = .82, .84; 
Nonacceptance: a = .86, .88; and Strategies: a = .85, .91). 

The Disruptive Behavior Disorders Rating Scale (DBD; 
Pelham, Gnagy, Greenslade, & Milich, 1992) is a 45-item 
parent- and teacher-report scale of 18 symptoms of ADHD 
(including nine inattention [IA] items and nine hyperactiv- 
ity/impulsivity [HI] items), 8 symptoms of oppositional 
defiant disorder (ODD), and 15 symptoms of conduct disor- 
der (CD). Raters use a 4-point scale (0 = not at all; 3 = very 
much), to rate how well each item describes their child, with 
higher scores indicative of greater symptom severity. The 
DBD has demonstrated evidence of reliability and validity 
(cf. Massetti et al., 2003). In the current middle school 
ADHD sample, parent-report data were collected and inter- 
nal consistency across all scales was acceptable (i.e., IA, a 
= .95; HI, a = .90; ODD, a = .92; CD, a = .85). 


The Emotion Regulation Index for Children and 
Adolescents (ERICA; MacDermott, Gullone, Allen, King, 
& Tonge, 2010) is a 16-item self-report measure of emotion 
regulation. Each item is rated on a 5-point scale (1 = 
strongly disagree; 5 = strongly agree), with higher scores 
indicative of greater emotion regulation. The ERICA con- 
tains three subscales: Emotional Control (seven items 
reflective of a low threshold for emotional excitability, 
impatience, defiance, and undercontrolled anger), 
Emotional Self-Awareness (five items focused on adapt- 
ability or flexible emotion management and a generally 
euthymic emotional style to which the individual is able to 
quickly return), and Situational Responsiveness (four items 
related to exhibiting socially appropriate emotional 
responses in social situations). It has demonstrated adequate 
reliability and evidence of validity (Bunford et al., 2014; 
MacDermott et al., 2010). The ERICA evinced adequate 
internal consistency in both the middle school and the high 
school ADHD samples across domains, respectively (i.e., 
Emotional Control: a = .86, .78; Self-Awareness: a = .62, 
.74), except for the situational responsiveness subscale, 
which demonstrated poor internal consistency in the middle 
school sample (a = .21), but adequate internal consistency 
in the high school ADHD sample (a = .64). 

The Multidimensional Anxiety Scale for Children 
(MASC; March, Parker, Sullivan, Stallings, & Conners, 
1997) is a 39-item self-report scale of anxiety symptoms 
across four domains (i.e., harm avoidance, physical symp- 
toms, separation, and social anxiety), administered to youth 
8 to 19 years old. Each item is rated on a 4-point scale (0 = 
never true about me; 3 = often true about me), with higher 
scores indicative of greater anxiety. The MASC has demon- 
strated evidence of reliability and validity (Baldwin & 
Dadds, 2007; March et al., 1997). Item-level data were not 
available. We used the total anxiety 7 score from the middle 
school ADHD sample (based on age and sex norms from a 
norming sample of >9,000 adolescents). 

The Index of Family Relations (IFR; Hudson, 1992) is a 
25-item measure of the magnitude and severity of personal 
and social impairment in the context of family relations. 
Each item is rated on a 7-point scale (1 = none of the time; 
7 = all of the time). Norms and standardization sample 
information reveal acceptable psychometric properties, and 
evidence of reliability and validity (Hudson, 1992). Self- 
report data were collected and the IFR demonstrated excel- 
lent internal consistency in the high school ADHD sample 
(a = .96). 

The Reynolds Adolescent Depression Scale-II (RADS-2; 
W. M. Reynolds, 2002) is a self-report scale of depressive 
symptoms in youth, across four domains: anhedonia/negative 
affect, dysphoric mood, negative self-evaluation, and somatic 
complaints. Each item is rated on a 4-point scale (1 = almost 
never; 4 = most of the time) with higher scores indicative of 
greater depression. The RADS-2 has demonstrated evidence 


Assessment 00(0) 


of internal consistency (a = 0.93), reliability, and validity 
(W. M. Reynolds, 2002). The full, 30-item version was 
administered to the middle school ADHD sample (but did not 
have item-level data available) and the short form (10 items) 
was administered to the high school ADHD sample (a = 
.87). In the current research, we used the total depression T 
score (based on age and sex norms from a norming sample of 
>2,500 youth). 

The Strengths and Difficulties Questionnaire (SDQ; 
Goodman, 1997) is a 25-item scale that has been validated 
for international use, with standardized norms across age 
groups and genders (Goodman, 2001). The SDQ includes 
five subscales: conduct problems, hyperactivity/inattention, 
emotional symptoms, peer relationship problems, and pro- 
social behavior. Items are rated on a 3-point scale (0 = not 
true; 2 = certainly true). The SDQ has demonstrated good 
specificity and moderate sensitivity in community samples 
in the prediction of mental health disorders (Goodman, 
Ford, Simmons, Gatward, & Meltzer, 2000). The scale is 
designed for community samples of the general population. 
In MTurk sample, all five subscales demonstrated adequate 
internal consistency (i.e., conduct problems: a = .74, 
hyperactivity/inattention: a = .79, emotional symptoms: a 
= .78, peer relationship problems: a = .66, and prosocial 
behavior: a = .77). 

The Impairment Rating Scale (IRS; Fabiano et al., 2006) 
was developed to assess areas of daily functioning (academic, 
social) that can be particularly problematic for adolescents 
with behavioral problems (specifically for adolescents with 
ADHD; Evans et al., 2013). Seven items are scored on a con- 
tinuous scale, converted to seven equal parts from (0 = no 
problem; 6 = extreme problem), with higher scores indicat- 
ing greater impairment. The psychometric properties of the 
IRS have been measured in both clinic and community sam- 
ples. The IRS demonstrates excellent temporal stability and 
evidence of convergent and discriminant validity. The IRS 
was administered to the MTurk and the middle school ADHD 
samples and the total item (i.e., “overall severity of your 
child’s problem in functioning and overall need for treat- 
ment’) was used in the current research. 


Data Analytic Plan. To maximize sample size, evidence for 
validity of the factor structure obtained in Study | was 
examined in Study 2 using the combined middle school 
and high school ADHD samples. CFA was conducted in 
Mplus 7.4 (Muthén & Muthén, 2012) using the weighted 
least squares estimator that is robust to nonnormality of 
categorical data. We also referenced available modifica- 
tion indices (MI), which reflect the degree to which the 
chi-square value would improve should a model modifica- 
tion be made. Model fit was examined using the x’/df 
ratio, root mean square error of approximation (RMSEA), 
and the comparative fit index (CFI). Conventionally, a 


x’/df ratio of 5:1 (Hooper, Coughlan, & Mullen, 2008; 
Wheaton, Muthen, Alwin, & Summers, 1977), an RMSEA 
<= .10, and a CFI = .90 indicate sufficient fit (Bentler, 
1990) and x’/df ratio of 2, RMSEA < .06, and CFI = .95 
indicate excellent fit (Hu & Bentler, 1999). Given that the 
DERS subscales are presumed to measure lower-order 
domains of a higher-order ED construct, the final four- 
factor correlated-traits CFA was compared with a higher- 
order CFA in which each factor from the correlated-traits 
model was specified as a lower-order factor and ED was 
specified as a higher-order factor. Both the correlated- 
traits and higher-order models were compared with a sin- 
gle-factor CFA in which the DERS was modeled as 
measuring a unidimensional ED construct. 

Following CFA, we assessed for age and gender differ- 
ences across DERS-P factors, using all three samples (i.e., 
online MTurk sample, middle school ADHD sample, and 
high school ADHD sample). We evaluated evidence of con- 
vergent, concurrent, and incremental validity of the DERS-P. 
To these ends, wherever possible, in the middle school 
ADHD sample and the high school ADHD sample, we 
examined bivariate correlations between the DERS-P and 
measures of the same phenomenon (i.e., emotion regulation) 
or measures of theoretically related characteristics (includ- 
ing those administered to the online sample, such as exter- 
nalizing and internalizing disorder symptoms, social 
impairment, family impairment, and overall impairment). To 
assess for evidence of incremental validity, we examined if, 
in regression analyses predicting outcomes associated with 
ED, DERS-P factors accounted for variance beyond that 
accounted for by the adolescent self-report DERS subscales 
collected in the middle and high school ADHD samples. 


Results and Discussion 


Confirmatory Factor Analyses. The four-factor structure 
obtained in Study 1 was assessed for cross-validation and 
model fit via CFA. Model fit was approaching, but not con- 
formably achieving acceptable levels across fit indices, 
x°(371) = 1,454.90, p < .001; RMSEA = .101 (90% CI = 
[.096, .107]); CFI = .90. MI indicated that DERS-P Item 22 
(“When my child is upset, he/she knows that he/she can find 
a way to eventually feel better.”) might best cross-load on 
Factor 3 (MI = 255.52) and Factor 4 (MI = 53.67). On 
examination of this item, which was indicated by the EFA 
to load onto Factor 1, its wording appeared inconsistent 
with the other items on that factor. Specifically, items on 
Factor | appeared to be related to catastrophizing secondary 
to negative emotions (e.g., end up feeling very depressed, 
feel overwhelmed, become out of control), whereas Item 22 
appears to reflect a tempered approach to handling upset 
emotions, inconsistent with catastrophizing. As such, Item 
22 was deleted and the CFA was conducted again. 
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Table 2. Means and Standard Deviations for the Factors of the DERS-P Across All Three Samples. 


Sample 


MTurk no special MTurk special needs, Middle school ADHD, High school ADHD, 


DERS-P Factors MTurk total, M (SD) needs, M (SD) M (SD) M (SD) M (SD) 
fl: Catastrophize 1.98 (0.82) 1.87 (0.73) 2.71 (1.01) 2.20 (0.79) 2.21 (0.84) 
f2: Negative 1.89 (0.77) 1.84 (0.72) 2.25 (0.98) 1.72 (0.71) 1.86 (0.77) 
Secondary 

f3: Attuned 2.55 (0.88) 2.48 (0.85) 3.01 (0.94) 2.88 (0.95) 2.83 (0.92) 
f4: Distracted 2.57 (1.07) 2.46 (0.99) 3.43 (1.21) 3.22 (0.98) 3.32 (1.00) 


Note. DERS-P = Difficulties in Emotion Regulation Scale—Parent-report form. t tests comparing factor means across each group were performed. The 
middle school ADHD and high school ADHD sample did not differ significantly on DERS-P factors, fl: t(282) = —0.12, p = .906; f2: (282) = —1.45, p 
= .148; f3: t(282) = 0.44, p = .661; f4: t(282) = -0.79, p = .433. MTurk total sample differed from the middle school ADHD sample across all factors, 
fl: t(1,054) = 2.34, p = .020; 2: t(1,054) = -1.97, p = .049; f3: t(1,054) = 3.23, p = .001; f4: t(1,054) = 5.22, p < .00I. The MTurk special needs 
subsample also differed from the middle school ADHD sample on factors: fl: t(184.92) = —4.21, p < .001, Cohen’s d = 0.61; f2: (189.30) = —4.06, p 
< .001, Cohen’s d = 0.66; but not f3 or f4. The MTurk sample differed from the high school ADHD sample across the following factors: fl: t(1,182) = 
3.77, p < .001, Cohen’s d = 0.28; f3: t(1,182) = 4.15, p < .001, Cohen’s d = 0.31; and f4: t(1,182) = 9.33, p < .001, Cohen’s d = 0.72; but not f2. The 
MTurk special needs subsample also differed from the high school ADHD sample on factors: fl: t(199.83) = —4.83, p < .001, Cohen’s d = 0.58; f2: 
t(189.61) = —3.93, p < .001, Cohen’s d = 0.48; but not f3 or f4. The MTurk no special needs subsample differed on all factors from the special needs 
subsample: fl: t(128.49) = 9.00, p < .001, Cohen’s d = 1.00; f2: t(128.70) = 4.60, p < .001, Cohen’s d = 0.51; f3: t(976) = 5.95, p < .001, Cohen’s d 


= 0.58; f4: (133.07) = 8.93, p < .001, Cohen’s d = 0.95. 


After deleting Item 22, model fit was marginally ade- 
quate considering all fit indices: x7(344) = 1,126.33, p < 
.001; RMSEA = .089 (90% CI = [.084, .095]); CFI = .93. 
Additionally, all items had a standardized factor loading 
estimate of 0.45 or greater, further supporting the conclu- 
sion of sufficient fit. All four factors maintained acceptable 
internal consistency (i.e., Factor | (Catastrophize): a = .91; 
Factor 2 (Negative Secondary): a = .89; Factor 3 (Attuned): 
a = .89; Factor 4 (Distracted): a = 84). Factor 1 was cor- 
related with Factor 2 (r = .66, p < .001), Factor 3 (r = .14, 
p = .010), and Factor 4 (r = .68, p < .001). Factor 2 was 
also correlated with Factor 4 (r = .40, p < .001), but not 
Factor 3 (r = .09, p = .157). Factor 3 was not correlated 
with Factor 4 (r = .003, p = .954). 

The final four-factor correlated-traits model was com- 
pared with a higher-order CFA in which the four factors in 
the correlated-traits model (1i.e., Catastrophize, Negative 
Secondary, Attuned, and Distracted) were specified as 
lower-order factors and ED was specified as a higher-order 
factor. A nonsignificant negative residual variance for the 
Catastrophize factor was fixed as zero. In the final model, 
model fit was marginally adequate across all indices: 
x’(347) = 1,036, p < .001; RMSEA = .084 (90% CI = 
[.078, .090]); CFI = .93. All four lower-order factors loaded 
significantly on the higher-order ED factor, though the 
loading for Factor 3 (Attuned) was smaller in magnitude 
(loading = .12) than those for the other three factors (load- 
ings => .65). Examination of the R’ indicated that the ED 
factor did not account for significant variance in the Attuned 
factor. Results of a x? difference test for the weighted least 
squares estimator indicated that the higher-order model did 
not result in significantly worse fit, y° DIFFTEST = 
4.40(3), p = .22. 


Both the correlated-traits and higher-order models were 
compared with a single-factor model in which the DERS 
was modeled as measuring a unidimensional construct of 
ED. For the unidimensional model, model fit was poor 
across all fit indices: x°(350) = 4499.47, p < .001; RMSEA 
= .204 (90% CI = [.199, .210]); CFI = .61. Results of x? 
difference tests for the weighted least squares estimator 
indicated that the single-factor model resulted in signifi- 
cantly worse fit compared with both the correlated-traits 
model, x? DIFFTEST = 720.20(6), p < .001, and the 
higher-order model, x” DIFFTEST = 604.22(3), p < .001. 


Evidence of Validity Across Factors. Mean factor scores were 
created for the four DERS-P factors in Study 1 and the 
ADHD samples in Study 2 (see Table 2 for means and stan- 
dard deviations). Notably, the more representative online 
MTurk sample demonstrated significantly lower parent- 
rated ED across factors than did the samples consisting of 
adolescents with ADHD (which did not significantly differ 
from each other). In addition to mean factor scores across 
samples, each factor was examined for differences, given 
adolescent age and gender. In the MTurk sample, age had a 
small negative association with ED on DERS-P Factor | 
(Catastrophize; r = —.08, p = .008) and Factor 2 (Negative 
Secondary; r = —.06, p = .049). Conversely, age was not 
associated with factors on the DERS-P in the ADHD sam- 
ples. Gender differences emerged in the MTurk sample and 
in the high school ADHD sample. In the MTurk sample, 
boys (M = 2.65; SD = 0.88) had higher scores on Factor 3 
(Attuned) than girls (WM = 2.43; SD = 0.86), (976) = 
—3,93, p < .001, Cohen’s d = 0.25. In the high school sam- 
ple, girls (M = 2.55; SD = 0.92) had higher scores on Fac- 
tor | (Catastrophize) than boys (M@ = 2.08; SD = 0.74), 
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Table 3. Bivariate Correlations Indicating Convergent Validity and Consistency Across Parent- and Self-Report of Emotion 


Regulation. 

DERS-P DERS- DERS- DERS- DERS- DERS- DERS- ERICA- 

factors Nonacceptance Goals Impulse | Awareness Strategies Clarity Control ERICA-SA — ERICA-SR 

Middle school ADHD sample 
fl as .06 gt 17 Jy .06 05 -.04 -.05 
f2 .03 -.16 -.01 .001 .06 -.03 .26* -.07 .02 
3 24* 03 =01 Te alt 24* = 7 =a =22 
f4 19 peli 18 AE} .27* .04 14 .08 01 

High school ADHD sample 
fl 09 .06 20s 13 || 137 -.08 =24"" -.03 
f2 .03 01 .04 .08 .07 226 .04 = 19% .04 
f3 -.04 -.03 -.03 Dye 05 allehe* -.08 =,23" -.134 
f4 05 10 .10 .07 .07 05 -.02 -.10 16 


Note. ADHD = attention deficit/hyperactivity disorder; DERS-P = Difficulties in Emotion Regulation Scale—Parent-report form; fl = Catastrophize; f2 
= Negative Secondary; f3 = Attuned; f4 = Distracted; ERICA = Emotion Regulation Index for Children and Adolescents; SA = Self-Awareness; SR = 
Situational Responsiveness. Shaded areas represent factors that share the most items from the adolescent DERS and the parent DERS-P. 


tb < 10. *p <.05. **p < Ol. 


t(172) = 3.25, p = .001, Cohen’s d = 0.56; and girls (VM = 
3.71; SD = 0.94) also had higher scores on Factor 4 (Dis- 
tracted) than boys (M = 3.21; SD = 0.95), (172) = 2.89, p 
= .004, Cohen’s d = 0.53. 

Next, we assessed for evidence of validity. First, we 
examined the degree to which the DERS-P exhibits evidence 
of concurrent validity as evaluated by bivariate correlations 
between the DERS-P and self-report emotion regulation on 
the DERS and ERICA (see Table 3). In general, the associa- 
tions between the DERS-P and the DERS and ERICA were 
small to medium in magnitude (at marginal to significant 
levels). The pattern of associations between the DERS-P 
factors and DERS subscales was generally comparable 
across the two samples, with an association between DERS-P 
Factor | (Catastrophize) and the DERS Impulse subscale, 
the DERS-P Factor 3 (Attuned) and the DERS Awareness 
subscale, DERS-P Factor 1 (Catastrophize) and DERS 
Strategies subscale, and the DERS-P Factor 3 (Attuned) and 
the DERS Clarity subscale. The DERS-P Factors 2 (Negative 
secondary) and 4 (Distracted) exhibited the least number of 
associations with DERS subscales, with a single statistically 
significant association in the high school and the middle 
school samples, respectively. 

The pattern of associations between the DERS-P and 
ERICA was not as comparable across the two samples. The 
DERS-P Factor 2 was positively associated with the ERICA 
Emotional Control subscale in the middle school ADHD 
sample, whereas DERS-P Factors 1, 2, and 3 were nega- 
tively associated with the ERICA Self-Awareness subscale 
in the high school ADHD sample. 

We then examined the degree to which the DERS-P 
exhibits convergent validity as evaluated by bivariate correla- 
tions between the DERS-P and other conceptually related 


characteristics across all three samples (see Table 4, organized 
in a way to aid comparison across samples). Analyses of the 
online MTurk sample revealed that the DERS-P factors gener- 
ally exhibited associations that are medium to high in magni- 
tude in the expected direction with the characteristics of 
interest (i.e., conduct problems, hyperactivity, emotional 
problems, peer problems, and impairment). In the middle 
school ADHD sample, many associations were significant 
(and, when not significant, many were moderate in magni- 
tude) and in the expected directions. In both the online MTurk 
and the middle school ADHD sample, parent-reported ADHD 
symptoms and conduct problems were associated with 
DERS-P factors. In the middle school ADHD sample, self- 
reported internalizing problems were not associated with 
DERS-P factors and in the high school ADHD sample, 
although these were associated, these associations were gen- 
erally small in magnitude. 

Finally, using the middle and the high school ADHD 
samples, we conducted hierarchical regression analyses to 
examine the degree to which the DERS-P factors exhibit 
incremental validity (i.e., variance accounted for) beyond 
that accounted for by factors of the self-report DERS in pre- 
dicting parent and adolescent ratings of psychopathology 
(i.e., ADHD, ODD, CD, anxiety, and depression symptoms) 
and impairment (i.e., self-reported family impairment and 
parent-reported overall impairment). To this end, we entered 
self-report DERS subscales in Step 1 and DERS-P factors 
in Step 2. The entry of DERS subscales and DERS-P factors 
were organized in accordance with number of overlapping 
items and magnitude of bivariate association (e.g., DERS 
Subscales 3 and 5 were added to Step 1, followed by 
DERS-P Factor 1, as the items on these factors overlapped 
most). To minimize the number of tests, we examined 
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Table 4. Assessing Convergent Validity for the DERS-P Factors and Theoretically Associated Variables Across the Three Samples. 


MTurk sample 
DERS-P factors SDQ:emo SDQ:peer SDQ:hyp SDQ:cndct IRS:T SDQ:pro 
fl 62° A6** 5 63 62° — 47 
f2 55 OFF AD A6** AGE -.28*°* 
f3 39 35** AG** 43° 3 —.50°** 
4 AR 29** 56° 36** 36°" —.27** 
Middle school ADHD sample 

MASC RADS-2 DBD:IA DBD:HI DBD:OD DBD:CD IRS:T 
fl .04 06 29* AQ** 65°" 35 55 
f2 201 09 33% A0** 34 22! AQ 
f3 10 .07 201 AS gt -.03 .00 
4 12 .07 35 AQ** 40** 221 4% 

High school ADHD sample 
BYI-II-A RADS-2-S IFR 

fl 29 23° 2 
f2 19 .18* 09 
f3 12 12 21 
f4 08 3 -.04 


Note. ADHD = attention deficit/hyperactivity disorder; DERS-P = Difficulties in Emotion Regulation Scale—Parent-report form; fl = Catastrophize; f2 
= Negative Secondary; f3: Attuned; f4: Distracted; SDQ:emo = emotional symptoms (parent report); SDQ:peer = peer relationship problems (parent 
report); SDQ:hyp = hyperactivity/inattention (parent report); SDQ:cndct = conduct problems (parent report); SDQ:pro = prosocial behavior (parent 


report); IRS:T = overall impairment due to behavior (parent report); MASC = total anxiety score (adolescent report); RADS-2-(S)= total depression 
t score (adolescent report); DBD-IA = ADHD inattention (parent report); DBD:HI = ADHD hyperactivity/impulsivity (parent report); DBD:OD = 
ODD symptoms (parent report); DBD:CD = conduct disorder symptoms (parent report); BYI-Il-A = Beck total anxiety score (adolescent report); 


IFR = total score from the Index of Family Relations (adolescent report). 
th < 10. *p < 05. *p <.01. 


regressions only in cases wherein the DERS-P factor exhib- 
ited a statistically significant bivariate association with the 
outcome. In most analyses, DERS-P factors accounted for a 
significant amount of variance (AR’) beyond the contribu- 
tion of the adolescent self-report DERS subscales (R°; see 
Table 5). In some cases, the DERS-P factors accounted for 
a large percentage of the variance, for instance, DERS-P 
Factor | (Catastrophize) accounted for 36% of the addi- 
tional variance in ODD symptoms beyond that of the 
Impulse and Strategies subscales of the DERS. 


General Discussion 


Our goal in this research was to evaluate the psychometric 
properties of a newly created parent-report form of a mea- 
sure of ED—the DERS-P form. Specifically, across two 
studies and three samples, we examined the (1) factor struc- 
ture, (2) internal consistency, and evidence of (3) conver- 
gent, (4) concurrent, and (5) incremental validity of the 
DERS-P. Generally, the DERS-P demonstrated acceptable 
reliability and evidence of validity. 

Regarding the factor structure of the DERS-P, a four- 
factor solution was obtained in Study | and confirmed with 


the ADHD samples in Study 2. The factors are Catastrophize, 
with items related to losing control over and feeling over- 
whelmed by negative emotions; Negative Secondary, with 
items related to a tendency to exhibit negative secondary 
emotions; Attuned, reflective of awareness or clarity of 
emotions; and Distracted, with items reflecting difficulty 
with concentrating or focusing, or completing tasks in the 
face of strong emotions. Each factor demonstrated accept- 
able internal consistency across studies. Interestingly, all 
factors correlated with each other (with at least a moderate- 
sized, r > .4 correlation) except for Factor 3 (Attuned), 
which correlated only and relatively weakly with Factor 1 
(Catastrophize). These results are consistent those obtained 
with the self-report DERS indicating that all self-report 
subscales are generally correlated, but the Awareness sub- 
scale is, albeit meaningfully related to the other subscales 
(Fowler et al., 2014; Giromini et al., 2012; Neumann et al., 
2010; Perez et al., 2012; Ritschel et al., 2015), typically 
exhibits a less strong correlation and sometimes exhibits no 
correlation. Although these results may be explained by 
there being commonalities among Factors 1, 2, and 4 (e.g., 
which seem to reflect the physiological and behavioral 
aspects of emotion regulation more than experiential or 
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Table 5. Assessing Incremental Validity of the DERS-P Relative to the Adolescent Self-Report DERS. 
Middle school ADHD sample 

MASC RADS-2 DBD:IA DBD:HI DBD:OD DBD:CD IRS:T 
DERS-P Step l* Step2 Step! Step2 Step! Step2 Step! Step2 Step! Step2 Step! Step2 Step! Step 2 
factors  (R) (AR) (R) (AR) (R) (AR) (R’) (AR) (R’)—s (AR) (R’)—s (AR) (R’), Ss (AR?) 
fl — — — — .04 .07* 01 LP 02 42% 01 1 2* — — 
f2 — — _ — 0! 10% 01 A ia 0! mE is .06 .26** 01 le 
f3 — — — — — — — — — — — — — — 
f4 — — — — .03 AS OL 18 01 LIS — _— 0! .20** 
High school ADHD sample 

BYI-II-A RADS-2-S IFR 
Step | (R?) Step 2 (AR?) Step | (R’) Step 2 (AR?) Step | (R?) Step 2 (AR?) 

fl 24 .04* 29" .02* — — 
f2 19 .03* .20** .03* — — 
f3 — — — — 27" Ol 
f4 — — — — — — 


Note. ADHD = attention deficit/hyperactivity disorder; DERS-P = Difficulties in Emotion Regulation Scale—Parent-report form; fl = Catastrophize; 

f2 = Negative secondary; f3: Attuned; f4: Distracted; IRS:T = overall impairment due to behavior; MASC = total anxiety score; RADS-2-(S)= total 
depression t score; DBD-IA = ADHD inattention; DBD:HI = ADHD hyperactivity/impulsivity, DBD:OD = ODD symptoms; DBD:CD = conduct 
disorder symptoms; BYI-II-A = Beck total anxiety score; IFR = total score from the Index of Family Relations. 

“When fl was entered on Step 2, f3 (Impulse) and f4 (Strategies) were entered for Step |; when f2 was entered on Step 2, fl (Nonacceptance) was 
entered for Step |; when f3 was entered on Step 2, f4 (Awareness) and f6 (Clarity) were entered for Step |; when f4 was entered on Step 2, f2 (Goals) 


was entered for Step |. 
th < 10. *p < 05. *p <.01. 


cognitive ones) relative to Factor 3 (e.g., which seems to 
highly reflect, if not exclusively, aspects of emotion regula- 
tion that are cognitive in nature) that account for such dif- 
ference, it will be important to examine whether, similar to 
the self-report DERS (Bardeen et al., 2012; Bardeen et al., 
2016; D. J. Lee et al., 2016), the reverse-coded items are 
driving the unexpected results. 

In examining the factor structure of the DERS-P, we also 
considered a higher-order model and a unidimensional 
model. The poor overall fit of the unidimensional model, 
and the poor fit in comparison with the four-factor corre- 
lated-traits and higher-order models, indicate that the 
DERS-P is not measuring a unidimensional construct. 
Although comparison of the model fit indices for the higher- 
order and correlated-traits model suggests that the higher- 
order model provides a more parsimonious and superior fit 
to the data, the Attuned factor did not demonstrate expected 
relations to the higher-order ED factor (i.e., weak loading 
on the higher-order ED factor). This finding is inconsistent 
with the hypothesis that all four factors of the DERS-P in 
the correlated-traits model represent lower-order aspects of 
a higher-order ED construct but is consistent with the results 
of Bardeen and colleagues (e.g., Bardeen et al., 2012; 
Bardeen et al., 2016; D. J. Lee et al., 2016). It is unclear 
whether this finding is attributable to the above-noted 


differences between the underlying phenomena measured 
by Factors 1, 2, and 4 and the Attuned factor (i.e., physio- 
logical and behavioral aspects of emotion regulation vs. 
cognitive aspects) or are a result of item format, rather than 
content. As such, although our results provide some evi- 
dence of a higher-order structure for the DERS-P, taking 
into account the magnitude of the path coefficients from 
lower-order to higher-order factors (specifically, of Attuned 
to the higher-order factor), it will be important to examine 
the higher-order structure of this measure using forward- 
coded items as in Bardeen et al. (2016), to rule out the pos- 
sibility that reverse-coded items are driving these effects. 
Given prior findings indicating that, relative to typically 
developing children (Maedgen & Carlson, 2000; Melnick 
& Hinshaw, 2000) or a community sample of youth 
(Bunford et al., 2014; MacDermott et al., 2010), youth with 
ADHD exhibit greater ED than youth without ADHD, this 
finding was expected in the current study (i.e., comparing 
the more representative online sample with the two samples 
of adolescents with ADHD). Indeed, our online MTurk 
sample was rated as less emotionally dysregulated than 
both our middle school and high school ADHD samples on 
the Attuned, Catastrophize, and Distracted factors, further 
confirming the growing body of work indicating ED is at 
least an associated feature of ADHD, across development, 
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measurement methods, and samples. However, the online 
MTurk sample was rated as exhibiting greater ED on the 
Negative Secondary factor than the sample of middle school 
youth with ADHD and as exhibiting comparable ED on this 
factor to the high school youth with ADHD. As depicted in 
Table 1, the Negative Secondary factor includes items per- 
taining to feeling angry or embarrassed about being upset. 
These negative appraisals of oneself following feeling upset 
may indicate a degree of metacognitive ability, such as an 
awareness and reflection that many youths with ADHD do 
not exhibit (Schroeder & Kelley, 2009). Indeed, the younger 
adolescents with ADHD were rated significantly lower on 
this factor—both their youth and their ADHD symptoms 
may in combination make them particularly prone to lack of 
awareness, insight, and/or remorse. 

There is literature indicating both the absence and the 
presence of age differences in ED or differential effects of 
age on the association between ED and other characteristics 
(Bunford et al., 2014; Bunford, Evans, et al., 2017; Bunford, 
Wymbs, et al., 2017; Graziano & Garcia, 2016; Zimmermann 
& Iwanski, 2014). Regarding findings suggesting the pres- 
ence of age differences in ED, there are general age-related 
increases in adaptive emotion regulation, though relative to 
other developmental phases, middle adolescence is charac- 
terized by the smallest emotion regulation strategy reper- 
toire (Zimmermann & Iwanski, 2014). 

In this research, there was a small negative association 
between age and ED in our more representative online sam- 
ple with the Catastrophize and Negative Secondary factors, 
but no such age-related decrease in ED was observed in our 
ADHD samples. These results may be mainly a factor of 
variability (i.e., wider age range in the representative sam- 
ple) but may also suggest that although normatively, observ- 
able ED decreases with age, either parent ratings are not 
sensitive to this change or observable ED does not decrease 
with age in the case of adolescents with ADHD. As such, 
our findings raise questions about parents’ ability to detect 
changes across development. Alternatively, or in addition, 
they may also suggest the gap in ED between youth with/ 
without ADHD may widen from early adolescence into and 
through late adolescence, increasing potential for functional 
impairments, such as social dysfunction (Bunford et al., 
2014; Bunford, Evans, Becker, et al., 2015). An exception 
to this may be aspects of ED measured on the Negative 
Secondary factor, as described above. 

As with age, prior findings suggest both the absence and 
the presence of gender differences in ED (Bunford et al., 
2014; Bunford, Evans, et al., 2017; Bunford, Wymbs, et al., 
2017; Graziano & Garcia, 2016; Zimmermann & Iwanski, 
2014). For example, there are gender differences with 
regard to primary motivation for use of emotion regulation 
strategies (i.e., downregulate negative vs. upregulate posi- 
tive emotions; Mitchell et al., 2012) and to individuals’ 
typically used emotion regulation strategies (Zimmermann 


& Iwanski, 2014). As in prior studies comparing men and 
women and boys and girls (with women/girls exhibiting 
greater ED on some of the factors, Bunford et al., 2014; 
Weinberg & Klonsky, 2009), gender differences emerged in 
the online MTurk sample and in our older ADHD sample 
(i.e., high school sample), but on different factors and in the 
opposite direction. In the MTurk sample, boys exhibited 
greater difficulties with awareness or clarity of emotions 
(Attuned), whereas in the high school sample, girls exhib- 
ited greater difficulty with losing control over and feeling 
overwhelmed by negative emotions (Catastrophize) as well 
as with concentrating, focusing, or completing tasks in the 
face of strong emotions (Distracted). It is of note that the 
MTurk sample was large and relatively well-balanced with 
regard to gender (51.2% boys), whereas the ADHD samples 
were predominantly boys (middle school ADHD: 75.6%; 
high school ADHD: 72.9%). The differences in attunement 
may be a product of a large enough sample of boys and girls 
wherein parents may attend to emotions in their children 
differently in a manner consistent with gender stereotypes 
(Chaplin, Cole, & Zahn-Waxler, 2005) and/or genuine dif- 
ferences in emotion expression across genders in adoles- 
cence (Chaplin & Aldao, 2013). The gender differences 
observed in the sample of older adolescents with ADHD 
should be considered with the population in mind. Boys are 
most frequently diagnosed with ADHD, yet available evi- 
dence indicates that girls with ADHD have at least as many, 
if not more, difficulties in peer relationships than do boys 
with ADHD (Mikami, 2010; Mikami & Hinshaw, 2003). It 
has been speculated that difficulties with the self-regulation 
of affect, motivation, and arousal are more likely to affect 
characteristic features of women/girls’ interpersonal inter- 
actions, such as emotional intimacy, focus on reciprocity, 
and verbal give-and-take than characteristic features of 
men/boys’ interactions (Mikami & Hinshaw, 2003). In 
addition, because ADHD is rarer in girls, a girl with ADHD 
may appear more deviant than a boy with ADHD (Mikami, 
2010). Even more generally, in line with the gender paradox 
perspective positing that the less-prevalent gender with a 
given disorder shows greater impairment than the more 
prevalent gender with the disorder (Eme, 1992), girls with 
ADHD may warrant greater concern. These hypotheses 
may apply to ED, in association with ADHD as well insofar 
as our findings may reflect parents with girls with ADHD 
and ED perceiving their children as more impaired across 
some domains, including in processes related to the experi- 
ence and expression of emotions, than parents with boys 
with ADHD and ED. 

Regarding evidence of convergent validity between the 
DERS-P and self-report DERS, not surprisingly, the DERS-P 
factors were generally more strongly associated with DERS 
subscales when they included overlapping items (see Table 
3). This differentiation in associations, wherein overlapping 
factors/subscales were more associated provides preliminary 
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evidence supporting the use of the DERS-P/DERS as parallel 
multi-informant reports of ED in future research. However, 
one exception to this was that the second DERS-P factor, 
Negative Secondary, was not associated with the DERS 
Nonacceptance subscale. Consistent with the above hypoth- 
esis speculating the Negative Secondary factor reflects lack 
of awareness, insight and/or remorse when upset, it may be 
that the lack of association between the DERS-P Negative 
Secondary factor and the DERS Nonacceptance subscale 
indicates that adolescents’ anger or embarrassment over 
their own behaviors related to having been upset is a private 
event they do not admit to parents. As a result, parental per- 
ceptions of this remorse may be different from adolescents’ 
self-reports. 

Regarding the relationship between the DERS-P and 
self-report ERICA, there is some evidence of convergent 
validity in the high school ADHD sample. Greater parent- 
rated ED (as indexed by tendency to lose control over and 
feel overwhelmed by negative emotions; Catastrophize, to 
exhibit negative secondary emotions; Negative Secondary, 
and to have difficulty with awareness or clarity of emotions; 
Attuned) was associated with greater self-rated ED (as 
indexed by adaptability or flexible emotion management 
and a generally euthymic emotional style to which the indi- 
vidual is able to quickly return; ERICA Self-Awareness). 
However, the magnitude of correlations is small, possibly 
due to the different aspects and manifestations that the 
DERS and ERICA measure. Together, data on concurrent 
validity between these parent- and self-report rating scale 
measures of ED indicate such validity depends on the spe- 
cific aspect of ED measured. 

Regarding concurrent validity, in the more representa- 
tive sample, DERS-P factors generally exhibited medium to 
high associations with characteristics of interest (1.e., con- 
duct problems, hyperactivity/inattention, emotional and 
peer problems, prosocial behaviors, and functional impair- 
ment) in the expected direction. In the middle school ADHD 
sample, parent-rated ODD symptoms were associated with 
each of the four DERS-P factors (rs ranging from .19 to .65) 
and parent-rated H/I symptoms were associated with all 
DERS-P factors except Factor 3 (Attuned; all rs .40). These 
correlation coefficients were large, but not so large as to 
indicate isomorphism. There was an interesting pattern of 
results in the high school ADHD sample, where both 
domains of self-rated internalizing pathology—anxiety and 
depression—were associated (with a small magnitude) with 
parent-rated tendency to lose control over and feel over- 
whelmed by negative emotions (Catastrophize) and to 
exhibit negative secondary emotions (Negative Secondary), 
but not the other two DERS-P factors. Of note, the weak 
relationships between anxiety and depression with the 
Negative Secondary factor are also consistent with the 
explanatory hypothesis speculating this factor is related to 
the tendency for self-reflection and remorse over one’s 


behavior exhibited when upset. Self-reported severity of 
personal and social impairment in the context of family 
relations was associated with parent-rated awareness or 
clarity of emotions (Attuned), but not the other three 
DERS-P factors. 

Finally, in the majority of regression analyses, DERS-P 
factors accounted for a significant amount of variance 
beyond the contribution of the self-report DERS in support 
of incremental validity of the DERS-P. This was true for 
analyses with other parent-report measures as the dependent 
variable as well as with self-report measures as the depen- 
dent variables, indicating that the additional variance 
accounted for was not solely a function of source. As such, 
using the DERS-P in isolation or in addition to the DERS is 
likely to provide important contextual information. Most 
adolescents are living with and relying on their parents and 
it is likely that a child’s ED has an effect on family function- 
ing (and vice versa; Bunford, Evans, & Wymbs, 2015). 
Understanding parents’ perspective of adolescent ED may 
be important, particularly considering that we observed that 
parent-reported ED accounts for considerable variance in 
ODD symptoms and other externalizing behaviors. Parents’ 
view of their child’s ED may be helpful in understanding the 
family context and how the family functions in reaction to 
the adolescent’s ED, ideally to inform interventions focused 
on the role of child ED in family dysfunction and the role of 
a potentially emotionally invalidating environment for a 
child exhibiting ED (Bunford, Evans, & Wymbs, 2015). 


Limitations and Directions for Future Research 


There are several limitations to this research that are impor- 
tant to note. First, in our CFA, model fit was marginally 
adequate, indicating it will be important to identify changes 
necessary to improve model fit, for example, as already and 
extensively discussed above by forward-coding all reverse- 
coded items (also see below). To improve model fit, we 
removed Item 22 arguing that although the EFA indicated 
for it to load onto Factor 1, its wording is inconsistent with 
the rest of the items. As already discussed, the issue may not 
be related to item content, but to it being reverse-coded and 
the forward-coded version may be more consistent with the 
“catastrophizing” content of the first factor. 

Second, it could be expected for parent-rated ED to be 
more strongly correlated with other parent-rated character- 
istics than self-rated ED simply due to rater/within-source 
bias. However, we remain optimistic about the promise of 
our findings of associations between parent-report and teen 
self-report, somewhat mollifying the concern of method 
bias. Nevertheless, future research may include measures of 
other characteristics of interest and measure them via self- 
report or ratings obtained from peers or teachers and expand 
to methodologies assessing other correlates or indices of 
ED—such as behavioral, cognitive, physiological, and 
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neural assessment to within-level-of-measurement bias 
(Morris & Cuthbert, 2012). 

Third, data were collected at posttreatment in the middle 
school ADHD sample, and there is reason to believe that at 
least one of the program components, the Interpersonal 
Skills Group (Evans, Schultz, DeMars, & Davis, 2011), 
could have some beneficial effects on ED, even in the for- 
mat in which it was delivered to the middle school ADHD 
sample (Bunford & Evans, 2017). Data having been col- 
lected at posttreatment may thus contribute to the smaller 
magnitudes of the observed associations between parent- 
reported ED and other indices of affective/behavioral func- 
tioning (also collected at posttreatment), due to broad 
treatment gains (Evans et al., 2016). However, these data 
likely did not affect evidence of construct validity, as what 
is important in this regard is that item variability and factor 
loading are similar regardless of intervention status or when 
data are collected. Relatedly, comparison of ED scores 
across the three samples should be made with caution as the 
MTurk and high school ADHD samples did not receive 
intervention from the researchers at the time of data collec- 
tion (but, as noted, the middle school ADHD sample did). 

Fourth, a multitude of tests were performed, raising pos- 
sibility of Type I error. Our results are encouraging by a 
generally consistent pattern of results across multiple sam- 
ples—especially considering the magnitude of effects, 
which has been argued to be as, if not more, meaningful and 
appropriate to interpret than p values (Baker, 2016; 
Wasserstein, 2016). 

Fifth, there are some limitations inherent to the original, 
self-report measure that we modified to create the parent- 
report form. Specifically, some of the limitations regarding 
the factor structure of the self-report DERS may also be 
applicable to the parent-report version; the problems that 
the retained reverse-coded items have been shown to cause 
with the self-report measure, may be plaguing the herein 
developed parent-report measure as well. All six of the 
Factor 3 items are reverse-coded, and, consistent with 
Bardeen et al. (2016), all of these items are made up of 
reworded self-report Awareness and Clarity items, poten- 
tially consistent with a method effect. A solution to explore 
in future research and thus a primary task in further validat- 
ing this parallel, parent-report form is to evaluate factor 
structure with all reverse-coded items forward-coded as in 
Bardeen et al. (2016). Another pertinent consideration is 
that the item content of the DERS is exclusively focused on 
dysregulation of negative emotions, despite data underscor- 
ing the importance and impairing potential of abnormalities 
in processing or regulating positive emotions (Bunford, 
Evans, & Wymbs, 2015; Bunford, Kujawa, et al., 2017; 
Okado & Mueller, 2016; Sjowall et al., 2015). 

Sixth, we included youth with ages spanning late child- 
hood through late adolescence. Although we statistically 
evaluated the effects of age in our analyses, longitudinal 


designs are needed to clarify the developmental course of 
the effects we observed or did not observe. Related, we did 
not assess and thus could not statistically account for puber- 
tal status, which might influence emotional processing (Silk 
et al., 2003). 

Finally, caution should be taken when generalizing the 
results of our study. For instance, although our CFA dem- 
onstrated marginally adequate model fit for factor struc- 
ture, it did not meet strict benchmarks for excellent fit. 
Our use of an ADHD-only sample to confirm factor struc- 
ture may have introduced some range restriction on vari- 
ance inherent in ADHD adolescent samples, and future 
cross-validation examinations should be performed to pro- 
vide further supporting evidence. Moreover, although the 
MTurk worker population is generally more representative 
than college students or other community samples, it is 
nevertheless a nonprobability sample (Chandler & 
Shapiro, 2016). MTurk workers better represent Internet 
users than non-Internet users and are generally younger 
and better educated than the general population (Chandler 
& Shapiro, 2016). Related, although we employed quality 
control checks and given the number of consented partici- 
pants relative to the number who completed the survey, 
confidence can be placed in sufficiently decreased likeli- 
hood of robots completing the online questionnaire, to 
ensure that not only “super-Turkers” (Bohannon, 2011) 
who often perform better than face-to-face participants 
(e.g., Hauser & Schwarz, 2016) but a representative sam- 
ple of parents complete our survey, we chose a somewhat 
lower approval rating threshold (85%) than is typical (i.e., 
95%; Bohannon, 2011). These considerations may limit 
the generality of our findings. 

Although our primary conclusions regarding the middle 
school sample do not involve these two subscales, caution 
should also be taken when interpreting any of the findings 
obtained with the middle school ADHD sample involving 
the self-report Awareness and Clarity subscales as those 
exhibited questionable internal consistency in that sample. 


Conclusion 


In conclusion, the current results provide support for the 
reliability and evidence of validity of the DERS-P for par- 
ent-report assessment of ED. Our findings demonstrate 
that although it will be a crucial next step in this line of 
research to examine whether some of the recently surfaced 
potential problems with the self-report DERS also apply to 
this parent-report version (i.e., related to the Awareness 
subscale and reverse-coded items; Bardeen et al., 2016), 
our results also suggest that the DERS-P is otherwise a psy- 
chometrically sound tool for the measurement of some 
parent-reported aspects of deficits in emotion regulatory 
capacities in adolescents aged between 11 and 17 years, 
including losing control over and feeling overwhelmed by 
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negative emotions; a tendency to exhibit negative second- 
ary emotions; lack of awareness or clarity of emotions; and 
difficulty with concentrating or focusing, or completing 
tasks in the face of strong negative emotions. Given the 
lack of an otherwise suitable parent-report measure of ado- 
lescent ED, the present study makes an important contribu- 
tion to the developmental and assessment literatures. 
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Notes 


1. Parent-report options are available to assess aspects of ED 
or characteristics that closely approximate it on many broad- 
band symptom scales (e.g., Behavior Assessment System 
for Children [BASC; C. R. Reynolds & Kamphaus, 1992], 
Behavior Rating Inventory of Executive Function [BRIEF; 
Gioia, Isquith, Guy, & Kenworthy, 2000], Child Behavior 
Checklist [CBCL; Achenbach, 1991]), but there are no com- 
prehensive measures of emotion regulation by parent report 
and the “emotion regulation” subscales on these measures do 
not have specific validity evidence supporting the “ED” com- 
ponent. Adding a valid parent-report measure of ED to the 
available assessments will enhance our ability to accurately 
measure the construct and partially address some of the limi- 
tations described above. 

2. See Evans et al. (2016) and Owens et al. (2018, November 
and IES grant #R305A140356) for detailed information on 
design and participants. 

3. In the middle school sample, more than 300 participants par- 
ticipated across 3 years; however, our sample size is smaller 
because the DERS-P was only administered to the third 
cohort (at Time-point 5, i.e., posttreatment). 
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